A New Approach for Dimensionality Reduction : Theory

This paper applies Whitney's embedding theorem to the data reduction problem and introduces a new approach motivated in part by the (constructive) proof of the theorem. The notion of a good projection is introduced which involves picking projections of the high-dimensional system that are optimized such that they are easy to invert. The basic theory of the approach is outlined and algorithms for nding the projections are presented and applied to several test cases. A method for constructing the inverse projection is detailed and its properties, including a new measure of complexity, are discussed. Finally, well-known methods of data reduction are compared with our approach within the context of Whitney's theorem. 1 The Data Reduction Problem Assume that the data set A of interest is a discrete sampling of a compact m-dimensional submanifold U embedded initially in an ambient vector space R q of high dimension (minimally q > 2m + 1 but it is generally much larger). Given the q-dimensional ambient space is redundant and an over-parametrization of the data the aim of reduction is to determine a new parametrization which more closely reeects the intrinsic dimension of the data. Our basic approach to this problem is to nd a smooth embedding of the data set B V such that the mapping G : A ! B is a diieomorphism. Furthermore, V is, by extrapolation, a re-embedding to U and resides in a space of lower dimension, i.e., V R p. Thus we seek a mapping G : U ! V which will retain the diierential structure of data set in its reduced space. In practice the mapping G will be determined by the data and speciically will be chosen s.t. the inverse mapping H is especially well-conditioned. The resulting composition of mappings H G should closely approximate the identity mapping. The ability to construct an inverse H is central to our approach in that it assures us that no data has been lost in the 1 procedure and that our re-embedded manifold V possesses all of the information contained in V. The approach to the reduction problem presented in this paper is motivated largely by the (easy) Whitney embedding theorem 8]. This theorem demonstrates that generically 1 a projection of any m-dimensional manifold is invertible provided the dimension of the range of the projection is no smaller that 2m + 1. Given this exibility, we propose to examine …